Synchronous wired-or ack status for memory with variable write latency

ABSTRACT

A memory controller comprises a command interface to transmit a memory command to a plurality of memory devices associated with the memory controller. The memory controller also comprises an acknowledgement interface to receive an acknowledgment status packet from the plurality of memory devices over a shared acknowledgement link coupled between the memory controller and the plurality of memory devices, the acknowledgement status packet indicating whether the command was received by the plurality of memory devices. In addition, the memory controller comprises a memory controller core to decode the acknowledgment status packet to identify a portion of the acknowledgement status packet corresponding to each of the plurality of memory devices.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/673,431, filed Nov. 4, 2019, now U.S. Pat. No. 11,101,393, issuedAug. 24, 2021, which is a continuation of U.S. patent application Ser.No. 15/369,244, filed Dec. 5, 2016, now U.S. Pat. No. 10,468,544, issuedNov. 5, 2019, which is a continuation of U.S. patent application Ser.No. 13/804,334, filed Mar. 14, 2013, now U.S. Pat. No. 9,515,204, issuedDec. 6, 2016, which claims the benefit of U.S. Provisional PatentApplication No. 61/680,611, filed Aug. 7, 2012, which is herebyincorporated by reference herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings.

FIG. 1 illustrates an ARAM Interface Subsystem Overview, according to anembodiment;

FIG. 2 illustrates a sample ARAM System Block Diagram, according to anembodiment;

FIG. 3 illustrates ARAM High-Speed IO Circuits, according to anembodiment;

FIG. 4 illustrates Very Low Swing-Near Ground Signaling (VLS-NG),according to an embodiment;

FIG. 5 illustrates an ARAM Protocol Format, according to an embodiment;

FIG. 6 illustrates a Single Device, Single Rank System (Dynamic WidthDisabled), according to an embodiment;

FIG. 7 illustrates a Two Device, Dual Rank System (Dynamic WidthDisabled), according to an embodiment;

FIG. 8 illustrates a Four Device, Dual Rank System (4:2 Dynamic Width),according to an embodiment;

FIG. 9 illustrates an Eight Device, Dual Rank System (4:1 DynamicWidth), according to an embodiment;

FIG. 10 illustrates Serialized CA Packet Waveforms, according to anembodiment;

FIG. 11 illustrates CA Packet Serialization and Deserialization,according to an embodiment;

FIG. 12 illustrates a DQ Write Transaction, according to an embodiment;

FIG. 13 illustrates a DQ Read Transaction, according to an embodiment;

FIG. 14A illustrates a Transmission of ACK packet relative to incoming,according to an embodiment;

FIG. 14B illustrates a Transmission of multiple ACK packets on a sharedACK link, according to an embodiment;

FIG. 14C illustrates different ACK packet codes for different memorydevices, according to an embodiment;

FIG. 14D illustrates the possible wired-OR results for ACK packet codes,according to an embodiment;

FIG. 15A illustrates an Example of 6-bit ACK packet having two 3-bitfields, according to an embodiment;

FIG. 15B illustrates an Example of a 12-bit ACK packet having four 3-bitfields, according to an embodiment;

FIG. 16 illustrates the effect of skew on ACK link timing, according toan embodiment;

FIG. 17 illustrates an example ACK timing diagram for a two-rank system,according to an embodiment;

FIG. 18 illustrates a sample ARAM System Block Diagram, according to anembodiment;

FIG. 19 illustrates ARAM Device Clocking, according to an embodiment;

FIG. 20 illustrates ARAM CPHY Clocking, according to an embodiment; and

FIG. 21 illustrates ARAM PHY Operating State Transitions, according toan embodiment.

DETAILED DESCRIPTION 1 Introduction

New non-volatile memory (e.g. resistive random-access memory (RRAM)) hassignificantly shorter latency for a write command (WRITE latency)compared to NAND-Flash. However, when using non-volatile memory, asuccessful WRITE operation is not always guaranteed and the latency of asuccessful WRITE operation is variable.

At the system level, WRITE throughput can be improved by interleavingWRITEs to multiple non-volatile memory (NVM) devices. Alternativemethods of transmitting a WRITE status acknowledgement (ACK) to the NVMcontroller can use one ACK pin per device, which increases system cost(e.g., increases the pin count), an asynchronous ACK per device that iswired-OR′d with ACKs from many devices, which assumes all devices arebusy when one device is busy (i.e., a performance penalty), or use apolling status bit in the NVM device, which can also lead to aperformance penalty. The present disclosure presents still anotheralternative for WRITE status acknowledgement, which is believed toenable lower pin count ACKs and high performance.

Embodiments of an Advanced Random Access Memory (ARAM) interfacearchitecture that includes the ACK channel and other novel features aredescribed herein. The ARAM interface architecture is designed, in atleast one embodiment, to enable the deployment of high-performance,low-latency, low-power, low pin count, low cost, non-volatile memory andstorage subsystems.

FIG. 1 illustrates an ARAM Interface Subsystem Overview, according to anembodiment. An ARAM interface architecture includes the technologyutilized to connect an ARAM Controller (AMC) 102, at the link or mediaaccess control (MAC) layer interface, to an equivalent link/MAC layerinterface in one or more ARAM non-volatile memory devices 104. The AMC102 may be referred to herein as the “memory controller core.” The scopeof this interface architecture includes the ARAM Controller PHY and IOcells (referred to subsequently as the CPHY) 106 and the ARAM device PHYand IO cells (ARAM PHY, or APHY) 108. A System-on-Chip (SoC) 110,containing the CPHY and the AMC (or other memory controller), will oftenbe referred to as the “Controller Component”, “ARAM Controller”, or“memory controller”.

In one embodiment, the ARAM CPHY 106 and APHY 108 communicate with eachother across a high-speed Very Low Swing-Near Ground (VLS-NG) signalinginterface. VLS-NG signaling operates in one design at speeds of up to 2Gbps, using very low voltage swings in order to minimize power. Areference clock (CK) and command-address (CA) and data (DQ) lanes allutilize the VLS-NG technology, while standard rail-to-rail CMOSsignaling may be used for chip enable (EN) lanes. An acknowledge/status(ACK) lane may use open-drain signaling.

In one embodiment, the CA lanes are utilized to deliver command andaddress packets from the CPHY 106 to the ARAM devices 104, while the DQlanes carry read and write data between the two. The number of CA and DQlanes used to communicate between the CPHY and the ARAMs are scalableparameters of the interface architecture. The number of CA and DQ lanesimplemented may be dependent upon the quantity and width of the ARAMdevices and the required CA and DQ bandwidth. In one embodiment, theCPHY is sized to support the appropriate number of CA and DQ lanes foreach system. All CA and DQ data transfers may be synchronous, meaningthat they are referenced to CK transitions. The FlexPhase architecturefrom Rambus may allow an extended range of per-pin transmit and receivedata phases for the DQ lanes, thereby eliminating the need for datastrobes and mitigating trace-length matching requirements, whichminimizes complexity, reduces signaling power, and eases the PCB layoutconstraints.

1.1 Features

The following are some features of the ARAM interface architecturedescribed herein.

-   -   Differential forwarded clock    -   Double-Data-Rate (DDR) DQ lanes    -   Single-Data-Rate (SDR) CA lanes    -   Per-Rank Enable (EN) link for efficient power management    -   Sideband ACK link for status and reporting of exceptional        conditions    -   Dynamic Width control and configuration support for capacity        expansion without bandwidth degradation.    -   Utilizes power efficient Very Low Swing-Near Ground (VLS-NG)        signaling    -   FlexPhase architecture for power efficiency and precise per-lane        phase alignment of the DQ links    -   Mitigates system trace length matching requirements

2.1 Interface Architecture Overview

This section provides an overview of the ARAM interface architecture. Itpresents an initial discussion on definitions utilized in thisdisclosure as well as a further discussion on the IOs, signaling,features, transactions, configurations and systems topologies supportedby the interface architecture

2.2 Pin Definitions

The ARAM interface utilizes the following pin types for normaloperation:

TABLE 1 Pin Usage Name Type Usage CK/CKB Diff VLS-NG Differential Clockfor ARAM subsystem CA Single ended VLS-NG Command and Address DQ Singleended VLS-NG Read and Write Data ACK Pseudo Open Drain Command Statusand Acknowledgement EN CMOS Device Enable VEXT Reference Voltage OutputReference Voltage VDD/GND Power Interface Power Supply

2.3 Logical Term Definitions

In one embodiment, an ARAM Memory Controller (AMC) communicates with anARAM Device through an ARAM interface channel. This channel is definedto contain lanes, links, and ranks. The AMC utilizes the ARAM ControllerPHY (CPHY) to communicate through the channel, while the ARAM deviceuses an ARAM PHY.

Within a channel, each physically and logically independent signal (orsignals) is referred to as a lane. For example, the differential signalpair CK and CKB can be referred to as the CK lane. Each logicallyindependent CA or DQ signal is a single-ended lane.

One or more logically related lanes can be grouped together and bereferred to as a link. For example, if two CA lanes form the means ofcommunicating commands on a given channel, they would constitute the CAlink for that channel. Similarly, logically related DQ lanes arecombined to form a DQ link.

Multiple ARAM devices may be attached to a single DQ link. If multipleARAM devices are attached to the same set of DQ lanes within a link, theARAM devices must be in different ranks. As each rank isenabled/disabled by its individual EN lane, the controller is able toeffectively manage which rank is drawing power and can service itsrequests. In one embodiment, the CK, EN, CA, DQ, and ACK links arecombined to form a single channel.

For example, as illustrated in FIG. 2, a single device ARAM x4 channelmay have a differential CK lane, a 2-lane wide CA link, a 4-lane wide DQlink, an EN lane, and an ACK lane. Each DQ lane in the ARAM x4 channelmay be able to transmit and receive data at up to 2 Gbps, for a totalbandwidth of 1 GB/s.

2.4 Dynamic Width

The ARAM interface architecture feature called Dynamic Width enablesadditional capacity, without the additional loading and signalinglimitations of simply adding ranks to increase capacity. Dynamic Widthallows multiple ARAM devices to be attached to a DQ link by distributingthem across subsets of DQ lanes within the link. To identify individualdevices, a sub-address field within the CA packet protocol is used toindicate the intended target. This method avoids the overhead ofproviding a per-device chip select lane on the controller.

2.5 Sample System Configuration

FIG. 2 contains a block diagram of a sample ARAM system configuration,according to an embodiment. In one embodiment, this configurationconsists of a controller component 210 (AMC 202 and its CPHY 206) and asingle x4 ARAM device 204 (ARAM PHY 208, link layer 212, and core 214),with an ARAM interface channel forming the communication means betweenthe two. This illustrated configuration utilizes 2 CA lanes and 4 DQlanes, providing 0.25 GB/s of CA bandwidth and 1 GB/s of DQ bandwidthwhen the data rates of 1 Gb/s for the CAs and 2 Gb/s for the DQs areused alongside a 1 GHz CK link. One ACK link serves as the status linkback from the device in this configuration, and a single EN lane is usedto enable the target device. Details of the appropriate protocol,signaling, and topology (for multiple ARAM device channels) aredependent upon the specific characteristics of the ARAM device, andadditional controller and system constraints.

In addition to the signals in the ARAM interface channel, there are twoother interfaces within the subsystem: one which connects the AMC 202 tothe CPHY 206, and one which connects the ARAM PHY 208 with Link Layer212. The signal counts shown on the internal interfaces illustrate thekey busses used at these interfaces, but do not show the complete set ofall signals needed between the AMC and CPHY as well as between the ARAMLink Layer and PHY, as these are implementation dependent and notdirectly relevant to the present disclosure. In one embodiment,additional external signals may be needed, based on the exact systemconfiguration selected. In normal operation, command packets from theAMC are serialized by the CPHY and sent to the ARAM across the CA link.Preamble and postamble fields may be used to frame these CA packets. TheCA packets themselves can include device address mask fields in order todirect commands to specific subsets of the attached ARAM devices.Similarly, write and read data to and from the ARAM device areserialized onto the bidirectional DQ link.

Note that the number of CA lanes is a flexible architectural parameter,and can be scaled depending upon the total required command/address linkbandwidth, command/address transport latency, and ARAM or controller pincount constraints. Similarly, the number of native DQ lanes is also aflexible architectural parameter.

TABLE 2 ARAM Interface Configurations Features Feature Supported byRecommended Features Class Architecture ARAMs ARAM Memory Controllers CAWidth x1/x2 x2 application dependent Native x4/x8 x4 ARAM applicationdependent DQ Width Dynamic x4 ARAM: x4 ARAM: N/A DQ Width x4, x2, x1 x4,x2 x8 ARAM: x8, x4, x2, x1

2.6 ARAM Interface Architecture Features 2.6.1 Operating Frequency

The ARAM system architecture is intended to operate in a wide range ofCA and DQ bit rates. It is initially targeted to provide a 1 GB/s DQlink (using 4 DQ lanes) with a 1 GHz forwarded clock (CK). The frequencyof the ARAM controller and ARAM core supporting 2 Gbps DQ data rate ispresumed to be 125 MHz. Consequently, the DQ serialization ratio at thelink layer interface for both the CPHY and the ARAM PHY is 16:1. As theCA lanes transfer data at 1 Gbps, the CA serialization ratio is 8:1.Note that in the ARAM system architecture, in order to minimizecontroller and device pin count while maintaining channel bandwidth, thetotal number of loads on the DQ bus is limited.

2.6.2 Link Definitions

Links in the ARAM system architecture channel are utilized as follows.(Note that CK, CA, and DQ links use VLS-NG signaling levels).

CK link: The differential clock link (consisting of the CK and CKBsignals) is driven from the CPHY to the ARAM devices. All CA and DQtransfers are referenced to CK transitions, as are the ACK packets. The(nominally) 1 GHz clock is generated by the CPHY's PLL and is amultiplied version of the reference clock (125 MHz nominal) feeding thePLL.

CA link: The CA link carries command and address information from theCPHY to all the ARAM devices connected to the interface channel (up to 4devices at maximum data rate). The architecture supports a x1 and a x2CA link. The link operates at a per-pin-data rate equal to the CK linkfrequency (e.g. a 1 GHz CK translates to a 1 Gbps CA per-pin data rate).A discussion of the command protocol can be found in section 4.1.

EN link: This link is used to enable the ARAM device, specifically, itsPHY and Link Layer. In order to successfully send and receive high speedVLS-NG signals across the CK, CA, DQ, and ACK links, EN must be assertedprior to and throughout the duration of the ARAM operation. To minimizechannel pin count, all devices within the same rank can receive the sameEN signal lane. Its proper use enables the multi-rank topology, which isuseful for capacity expansion. The EN signal is presumed to beasynchronous, and is the only signal within the channel that usesfull-swing CMOS signaling in normal operation.

DQ link: The bidirectional DDR DQ link carries data between thecontroller and the ARAMs. Four-bit and 8-bit wide DQ links aresupported. Alongside a forwarded-clock topology, FlexPhase technology isused to maximize READ and WRITE timing margins for the DQ lanes.

ACK link: is used to return routine status information and exceptionalcondition notification from the ARAM devices to the CPHY. The specificinterconnect, signaling, protocol, and timing appropriate for the ARAMdevice characteristics is discussed in the ACK section of thisdisclosure.

2.6.3 ARAM Interface Architecture IOs

The controller and ARAM I0 cells may utilize a very low swing-nearground differential driver and receiver for the clock link. Very lowswing-near ground, single-ended drivers and receivers may be used forthe DQ and CA lanes. At 2 Gb/s with 2 loads per DQ lane, the receiversare un-terminated to minimize signaling power. FIG. 3 shows how thesehigh-speed IOs are connected for the CK 310, CA 320, and DQ 330 links,according to one embodiment. In this embodiment, the DQ lanes arebidirectional, while the CA and CK lanes are unidirectional from thecontroller to the ARAM.

2.6.4 VLS-NG Signaling

As noted, the ARAM interface channel utilizes Very Low Swing-Near Groundsignaling (VLS-NG) for its high-speed TO links, in order to optimizepower consumption. This signaling technology uses a voltage supplied bythe controller, VEXT, as reference for the VOH signal level. The rangeof VEXT/VOH is expected to be 250-350 mV, while VOL is defined as OmV.

For the single-ended signals within the channel, a reference voltage isutilized as the receiver sampling threshold and is set to VREF=VEXT/2.For the differential CK link, a differential peak-to-peak signal swingof 2*VOH is utilized. FIG. 4 provides a graphic representation of thevoltage waveforms present on the VLS-NG links.

The FlexPhase technology allows for different timing on each of the DQlanes. These links are individually calibrated, internal to the CPHY, inorder to provide optimal timing of the eye relative to CK. The CPHYcontains circuitry to calibrate the DQ lanes in both the write and readdirections. Consequently, the externally observed timing of CK relativeto DQ is not necessarily as shown in FIG. 4, which depicts the timing asseen at the samplers in the ARAM, or in the CPHY after link calibration.

2.6.5 ARAM Interface Transactions

A sample sequencing of enable signals, command/address packets, datapackets, and acknowledge/status signals is shown in FIG. 5. Thesub-fields within the command/address packet are devoted to command type(e.g. read or write, device select, register or data address, datapacket length, write mask, etc.) are described elsewhere. FIG. 5illustrates the serialization of 48-bit command/address packets, forboth back-to-back 64B read data transactions and a 64B write datatransaction.

2.7 Supported ARAM Device Configurations 2.7.1 ARAM Core Organizations

Various core organizations can be supported by the ARAM interfacearchitecture. An ARAM Controller can support the full architecturalrange, but the verification may be focused, based on system requirementsor current ARAM implementations. At minimal power and area overhead, amemory controller with a wider range of architectural support eases themigration path to newer ARAM devices, as they become available.

2.7.2 DQ Dynamic Width Configuration

In one embodiment, the ARAM system architecture supports Dynamic Widthconfiguration, allowing capacity expansion without degrading the channelbandwidth due to additional loading. When Dynamic Width data pathmultiplexing is enabled, the external width of the ARAM DQ link isreduced, and ARAM device data traffic is carried on a subset of thedevice's DQ pins. For a given system memory width, more memory devicescan be connected to the CPHY, resulting in expanded memory capacity.

Table 4 lists the DQ configurations supported by the ARAM interfacearchitecture. While the ARAM interface architecture supports the fullrange of possible Dynamic Width connectivity and configurability, boththe complexity of the connectivity discovery sequence the controllermust execute during initialization and the datapath logic within theARAM device will be simplified if only the minimal initialconfigurations are implemented.

TABLE 4 ARAM Dynamic Width DQ Configurations Supported ArchitectureRange Native Dynamic Width Configuration DQ DQ DQ Width Signals WidthActive DQ Signals x4 ARAM DQ[3:0] x4 DQ[3:0] x2 DQ[3:2] DQ[1:0] x1 DQ[3]DQ[2] DQ[1] DQ[0] x8 ARAM DQ[7:0] x8 DQ[7:0] x4 DQ[7:4] DQ[3:0] x2DQ[7:6] DQ[5:4] DQ[3:2] DQ[1:0] x1 DQ[7] DQ[6] DQ[5] DQ[4] DQ[3] DQ[1]DQ[0]

3 ARAM System Configuration

The ARAM interface architecture provides for a broad range ofconfigurations, in order to meet application bandwidth, capacity, andlatency requirements. This section illustrates a range of various memorysystem configurations that can be constructed using the ARAM interfacearchitecture.

3.1 Example System Configurations

Table 5 lists a range of system configuration choices: each row is aspecific system capacity (assuming 16 Gb ARAM devices), each column isthe total number of ARAMs, the number of ranks, and the dynamic widthmode. Note that the default for the Dynamic Width mode is the nativedevice width (x4).

TABLE 5 Capacity, Ranks, and Dynamic Width Options ConfigurationCapacity ARAMs Ranks ARAM Dynamic Width 1 2 GB 1 1 x4 2 4 GB 2 2 x4 3 8GB 4 2 x2 4 16 GB  8 2 x1

3.2 Single Device, Single Rank

FIG. 6 shows a x4 CPHY 610 that is connected to a single x4 ARAMcomponent 620, according to an embodiment. This figure illustrates howthe components of the memory system logically interact, without goinginto the details of each connection.

3.3 Two Devices, Dual Rank

FIG. 7 shows a x4 CPHY 710 connected to two x4 ARAM devices 720 inseparate ranks, according to an embodiment. The figure illustrates howthe components logically interact, without going into the details of theindividual connections. In one embodiment, the two ARAM components 720share the same CK, CA and DQ links. Per-rank PHY power-down state ismanaged with separate EN lanes. CA packets include a sub-address fieldto target individual ranks. In this example, the controller hasprovisioned an ACK status lane for each rank.

3.4 Four Devices, Dual Rank, 4:2 Dynamic Width

FIG. 8 shows a x4 CPHY 810 connected to a four x4 ARAMs 820, accordingto an embodiment. This figure illustrates how the components of thememory system logically interact, without going into the details oftheir connections. This configuration utilizes the Dynamic Widthfeature, which doubles the system memory capacity with the same systembandwidth. The x4 ARAMs 820 operate in x2 mode, with each ARAM providinghalf of the x4 wide data expected by the CPHY 810.

In one embodiment, the four ARAM components 820 share the same CK and CAlinks. Each ARAM component is configured in 4:2 dynamic width mode. Eachof the two ARAM components (connected to different DQ lanes) within arank share the same EN link. The ranks are selected by separate ENlanes, so that they can be independently placed into PHY power-downmode. During normal operation, the DQ data may be striped across the twocomponents within the selected rank; the two components are operated inlockstep. Sub-address fields within the CA packet are used to addresscommands to individual components. In this example, one ACK status laneis shared by multiple memory devices within each rank.

With the Dynamic Width feature enabled and configured at power-on, onlyhalf of the DQ lanes on each ARAM device are active; the other half ofthe lanes are not used. This allows 2 ARAMs to replace a single ARAM;maintaining the same DQ loading and bandwidth as with a single ARAM, butwith twice as much system memory capacity.

3.5 Eight Devices, Dual Rank, 4:1 Dynamic Width

FIG. 9 shows a x4 CPHY 910 connected to eight x4 ARAMs 920, according toan embodiment. This figure illustrates how the components of the memorysystem logically interact, without going into the details of theirconnections. This configuration utilizes the Dynamic Width feature,which provides 8 times the system memory capacity of a single ARAMdevice. The x4 ARAMs 920 operate in x1 mode, with each ARAM providingone quarter of the x4 wide data expected by the CPHY 910.

The eight ARAM components 920 share the same CK and CA links. Each ARAMcomponent is configured in 4:1 dynamic width mode. All four of the x4ARAM components (connected to different DQ lanes) within a rank sharethe same EN link. The ranks are selected by separate EN lanes, so thattheir operating power can be independently controlled. During normaloperation, the DQ data is striped across the four components within theselected rank; the four components are operated in lockstep. Sub-addressfields within the CA packet are used to address commands to individualcomponents. In this example, one ACK status lane is also shared bymultiple devices 920 within each rank.

With the Dynamic Width feature enabled and configured at power-on, onlyone of the DQ lanes on each ARAM device is active and the other threelanes are not used. This allows 4 ARAMs to replace a single ARAM withina rank; maintaining the same DQ loading and bandwidth as with a singleARAM, but with four times as much memory capacity. As this configurationhas some ramifications with respect to signal integrity and hence databandwidth, less than maximum data rates would be advised.

4 ARAM Interface Protocol and Serialization

This section describes the protocol used to communicate between theController and ARAM devices. It also describes the serialization of CApackets from the Controller PHY Link Layer interface to ARAM interfacepins, their subsequent deserialization in the ARAM PHY, and theirdelivery to the ARAM PHY Link Layer interface.

4.1 CA Protocol

When an ARAM device has been enabled with the assertion of the EN signaland the clocks started, the ARAM PHY will snoop the CA link searchingfor the preamble that delimits the start of a CA packet, as shown in thewaveforms of FIG. 10.

In one embodiment, the minimum preamble duration is two CK cycles.Detection of malformed preambles can result in assertion of the ACKsignal, and ignoring the remainder of a CA packet until the duration ofthe system defined maximum length command packet has been exceeded.Detection of a well formed preamble allows proper framing of thedeserialized CA packet that the ARAM PHY outputs to the ARAM Link layer.Note that during the time interval after the assertion of EN and clocks,and prior to the detection of a preamble by the targeted device, theARAM PCLK divider is free running.

Following the preamble, 8 bits (four CK cycles with two CA lanes) ofcommand address (CADDR) information are serialized. Link layer logicwithin each ARAM will compare CADDR to the DEVADDR (device address)established during the initialization/configuration procedure todetermine whether or not the command is targeted for the device.Partitioning the 8 bit CADDR field into two 4-bit subfields allows thebroadcasting of command packets to up to 4 devices in up to 4 ranks.This use of the CADDR field enables the sharing of the CA link amongmultiple devices in different ranks, minimizing the total controller pincount. The ARAM PHY may choose to unconditionally accept commands thathave CADDR=8′h00. It is the responsibility of the ARAM controller toschedule packets and transactions such that DQ driver conflicts areavoided.

The next 4 bits (2 CK cycles with two CA lanes) of the CA packet containthe command (CMD) field, allowing up to 16 distinct commands to bedefined. Immediately following are 15 CK cycles (30 bits with two CAlanes) of command payload, whose length, contents, and semantics arespecific to the particular command. CA packets are terminated with a 1CK cycle postamble. The total CA packet length—incuding the pre andpostambles—is 24 CK cycles (48 bits total) with two CA lanes. With an8:1 serialization ratio, the Link-PHY interfaces will provide andreceive CA packets in 3 PCLK cycles.

One embodiment of serialization of CA packets, from the ARAM ControllerLink-CPHY interface, across the ARAM interface, and afterdeserialization, delivered to the ARAM APHY-Link interface, is shown inFIG. 11. The initial detection of a preamble after exiting the PHYPowerdown state is used to establish the phase of the APHY PCLK, whichframes the deserialized CA and DQ data received by the ARAM device, andcan remain unaltered as long as EN is asserted.

The ARAM Controller can ensure that the ARAM PHY CK and CA receivers andclock trees used to sample CA lanes have reached a stable operatingcondition prior to instructing that the CPHY serialize CA packets acrossthe CA link. By idling the CPHY Link CA bus at 16h′0000, the packetizedprotocol with preamble detection will provide further protection againstthe consequences of CK glitching and sampling errors that can occur whenexiting the PHY Powerdown state.

4.2 DQ Write Transactions

FIG. 12 shows the timing for a write transaction in an ARAM system witha x4 Controller component connected to a x4 ARAM device, according toone embodiment. The timing diagram assumes that a two lane CA link(CA[1:0]) is used to convey the write command and address information tothe ARAM. It also assumes that the ratio of the frequencies of theController and ARAM link layer clocks (PCLK) to the ARAM interface CK is8:1. With DDR signaling, the DQ serialization ratio is 16:1, and with 4DQ lanes, the natural width and granularity of PCLK domain DQ data is 64bits.

The write transaction command from the ARAM Controller is firstserialized in the CPHY and then driven onto the CA lanes with a delay ofsome period of time from the assertion of the parallel command to theassertion of the first serial CA bit. This delay includes aclock-domain-crossing component, from the internal parallel clock (PCLK)domain to the external clock (CK) domain.

At a time after driving the WR command, the ARAM Controller drives theWrite Data (CPHY Link Q[63:0]). This data is then serialized by the CPHYat a 16:1 serialization ratio and driven onto the DQ links with a delayfrom the assertion of the first parallel data word to the assertion ofthe first serial DQ bit.

The phase of the ARAM PCLK used to frame deserialized CA data isestablished by the reception of the first CA packet received after theassertion of EN. The same phase of PCLK must also be maintained to framedeserialized DQ data. Consequently, it is the responsibility of theFlexPhase logic within the CPHY to launch data from the controller atthe proper time, accounting for the cumulative static and dynamic pathlength differences between CA and DQ lanes, such that properly frameddeserialized DQ data is delivered by the ARAM PHY to the ARAM Link Layerinterface. Details of the calibration sequence of operations necessaryto determine the proper delays within CPHY, to be applied on a per-ARAMdevice and per-rank basis, are described in Section 8.

Due to the limited endurance and variable write timing of ARAM cells, inone embodiment, all write DQ transactions are acknowledged. Positiveacknowledgements of successfully committed write transactions arecommunicated back to the host in order to release write data that hasbeen buffered upstream. Negative acknowledgement of unsuccessful writetransactions can similarly be communicated back to the host in order toallow retrying the operation. The positive or negative acknowledgementinformation is returned to the host at a pre-determined time through theACK lane. The ACK protocol is discussed in a subsequent section of thisdisclosure.

4.3 Read Transaction

FIG. 13 shows the timing for a read transaction in an ARAM system with ax4 Controller component connected to a x4 ARAM device, according to anembodiment. The timing diagram assumes that a two lane CA link (CA[1:0])is used to convey the read command and address information to the ARAM.It also assumes that the ratio of the frequencies of the Controller andARAM link layer clocks (PCLK) to the ARAM interface CK is 8:1. With DDRsignaling, the DQ serialization ratio is 16:1, and with 4 DQ lanes, thenatural width and granularity of PCLK domain DQ data is 64 bits.

The read transaction command from the ARAM Controller is firstserialized in the CPHY and then driven onto the CA lanes with a periodof delay from the assertion of the parallel command to the assertion ofthe first serial CA bit. This delay includes a clock-domain-crossingcomponent, from the internal parallel clock (PCLK) domain to theexternal clock (CK) domain. At some time after receiving the RD command,the ARAM device link layer drives the Read Data (APHY Link Q[63:0]).This data is then serialized by the APHY at a 16:1 serialization ratioand driven onto the DQ links with a delay from the assertion of thefirst parallel data word to the assertion of the first serial DQ bit.

The phase of the ARAM PCLK used to frame deserialized CA data isestablished by the reception of the first CA packet received after theassertion of EN. The same phase of PCLK must also be maintained to framedeserialized DQ data. Consequently, it is the responsibility of theFlexPhase logic within the CPHY to sample data received from the ARAM atthe proper time, accounting for the cumulative static and dynamic pathlength differences between CA and DQ lanes, ARAM read latency, and allclock domain crossings, such that properly framed deserialized DQ datais delivered by the ARAM Controller PHY to the ARAM Controller LinkLayer interface. Details of the calibration sequence of operationsnecessary to determine the proper delays within CPHY, to be applied on aper-ARAM device and per-rank basis, are described in Section 8. It isassumed that the latency of read transactions is deterministic, and thatacknowledgement via the ACK lane is not needed.

5 ACK Link Signaling and Protocol 5.1 Overview

In one embodiment, the ARAM interface includes a high-bandwidth,low-latency ACK link to provide a mechanism for ARAM devices to sendstatus of outstanding operations back to the controller. For example, anARAM device may need to notify the controller immediately after a WRITEoperation has been successfully completed.

5.2 ACK Link Signaling

As described in Section 3 (ARAM System Configuration), each ARAM devicemay have one dedicated ACK pin. ARAM devices in the same rank can sharean ACK link in order to reduce the controller pin count.

In one embodiment, wired-OR signaling is used in the shared ARAM link.Each ARAM device may drive an open-drain pull-down transistor totransmit a logic “1”. The ARAM device may place the pull-down transistorin high-impedance (OFF) state to transmit a logic ‘0’. The ARAMcontroller is expected to provide termination in order to resolve theACK signal. In one embodiment, each ARAM device drives the pull-downtransistor to transmit a logic “1” as part of the ACK codes describedbelow. If one ARAM device is pulling the line down, the line may have afirst expected voltage value. If a second ARAM device is pulling theline down at the same time, the line may have a second expected voltagevalue, which is lower than the first expected voltage value. Since bothARAM devices are pulling the line in the same direction to transmit alogic “1” and not pulling the line at all to transmit a logic “0”, thesituation does not exist where two device are pulling the line indifferent directions, possibly canceling each other out and causinginaccurate readings at the controller.

In one embodiment, the nominal signaling rate of an ACK link is ¼ of theCA signaling rate. For example, a system with 1 Gbps CA signaling ratemay have a 250 Mbps ACK signaling rate.

5.3 ACK Link Protocol

In one embodiment, an ARAM device sends back operation statusinformation to the controller in the form of an ACK status packet. Thetransmission of an ACK status packet may be aligned to the clock signalCK. The length of an ACK status packet may be the same as the CA packetlength. Because the ACK signaling rate is ¼ of the CA signaling rate,the number of bits in an ACK packet may be ¼ of the number of bits in aCA packet (e.g., for 24-bit CA packet, each ACK packet may be 6-bitslong). FIG. 14A illustrates an ACK packet transmission by an ARAM devicerelative to incoming CA. The time between a CA preamble and the start ofan ACK packet is labeled as tCMD-ACK. In one embodiment, there are twocomponents of tCMD-ACK, as described below.

The first component is N*CA PACKET LENGTH, where N is an integer.Because this component is quantized to the CA packet length, it mayscale with CK frequency. The integer N may be a variable. For example,if the command payload in the CA packet is a WRITE command, N may dependon the write latency of this particular instance. In the exampleillustrated in FIG. 14A, N=3.

The second component is some period of timing uncertainty, which isattributable to on-chip clock distribution and other causes of skew.

FIG. 14B illustrates a transmission of multiple ACK packets on a sharedACK link, according to an embodiment. In one embodiment, each ARAMdevice is assigned a unique bit-sequence for use in ACK packets. Theunique bit sequence may be based on a unique identifier of each device,for example. Each ARAM device can then transmit the unique bit sequenceas an ACK packet to indicate the status of write operations received ina CA packet. The multiple packets can be wire-OR′ d on the same ACK linkand sent to the controller. The memory ARAM controller can decode thebit sequences to determine which ACK packet is received from which ARAMdevice. As shown in FIG. 14B, the ACK1 packet and ACK3 packet are sendon the ACK link 1450 at the same time. In conventional systems, the twoACK packets would collide. In the present embodiment, however, theunique bit sequences in ACK1 and ACK2 allow the ARAM controller tocorrectly decode the two ACK packets send over ACK link 1450 atapproximately the same time.

FIG. 14C illustrates different ACK packet codes for different memorydevices, according to an embodiment. In one embodiment, device 1 has aunique set of codes to signify pass, fail and NOP (no operation, i.e.,no indication of acknowledgment is being sent). The codes may be acombination of logic values (in this case a 3-bit value for each). Asillustrated, the codes are different from the code associated withdevice 2. In other embodiments, the codes may include some other numberof bits or may have different logic values.

FIG. 14D illustrates the possible wired-OR results for ACK packet codes,according to an embodiment. In one embodiment, two of the codes shown inFIG. 14C (i.e., one from each device), may be send on the ACK link atthe same time. The codes may be wired-OR'd together to form a singlepacket. For example, the first received result 1480 includes the value1,2,2. The memory controller can decode this value to determine thecorresponding codes for each memory device. For example, the receivedresult 1,2,2 may decode to Device 1=0,1,1 (i.e. Pass) and Device 2=1,1,1(i.e., Pass). Thus, both devices acknowledge the successful receipt ofthe command instruction. The remaining possible results in FIG. 14D mayindicate other combinations of codes from Device 1 and Device 2.

Because multiple devices in the same rank may share an ACK link andbecause ARAM devices may have variable write latency (i.e., the variablewrite latency is quantized to CA packet length as discussed above), ACKpackets from the devices in the same rank may collide. To resolve thisissue, an ACK packet may be divided into N fields (i.e., one field foreach device) to avoid collision. In one embodiment, the assignment ofACK fields in an ACK packet is based on the Device ID. FIG. 15Adescribes an example of a system having two devices per rank. In theillustrated embodiment, there are two fields in one ACK packet (i.e.,Device #1 ACK Field and Device #2 ACK Field), each field consisting of 3bits. In this case, each ARAM device can transmit up to 3 status bits inan ACK packet.

FIG. 15B illustrates an Example of a 12-bit ACK packet having four 3-bitfields, according to an embodiment. In one embodiment, each of the 3-bitfields corresponds to a different ARAM device. Each ARAM device maytransmit a full 12-bit ACK packet as shown. However, each ARAM devicemay only use a designated 3-bit field to include acknowledgmentinformation, while the rest of the packet is full of values that do notperturb the ACK lane (e.g., all 0's). In one embodiment, bits 0-2correspond to Dev1, bits 3-5 correspond to Dev2, bits 6-8 correspond toDev3, and bits 9-11 correspond to Dev4. In one embodiment, when a deviceacknowledges a successful write, a logic value of 110 may be included inthat device's corresponding ACK field. A logic value of 100 may indicatean unsuccessful write and a logic value of 000 may indicate noacknowledgment. In other embodiments, the ACK packet may have some othernumber of bits, some other number of fields, may have the fieldsassigned to different ARAM devices or may use different logic values.The four ACK packets (i.e., one from each ARAM device) may be wired-OR'dtogether to form the received wired-OR packet. The ARAM controller canexamine the received wired-OR packet to identify what logic values areincluded in each ACK field to determine whether a particular operationwas successful on each ARAM device.

Because a single ACK link is shared by multiple devices using wired-ORsignaling, there may be degradation in the quality of the ACK signalreceived by the controller. The degradation may be caused by on-chipclock distribution skew (and potentially package/board routing skews insome systems) among the devices sharing the ACK link. This issue isshown in FIG. 16. In one embodiment, in order to achieve an optimumsampling point, an ACK link calibration may be performed duringinitialization.

In two-rank systems, each rank may have its own dedicated ACK link asdescribed above in Section 3 (ARAM System Configuration). An ACKprotocol timing diagram for a two-rank system is described in FIG. 17.As shown in FIG. 17, the CA packet stream 1700 includes multiplecommands (e.g., CMD #1, CMD #2, CMD #3, CMD #4). CMD #1 and CMD #2 aredirected toward ARAM devices in rank-0 and CMD #3 and CMD #4 aredirected to ARAM devices in rank-1. In one embodiment, the ARAM ranksare constrained in two ways. First, ACK packets are transmitted on a CApacket boundary. Second, ACK packets are returned in the same order asthe CA packets were received at the memory device. In one embodiment,however, the particular CA packet boundary on which the ACK packet isreturned is flexible. The boundary can change depending, for example onthe type of command in the CA packet. Thus, the first ACK packet (i.e.,ACK #1) is transmitted on the boundary between CMD #3 and CMD #4 (adelay of 3 packet lengths). The third ACK packet (i.e., ACK #3),however, which is sent by an ARAM device of rank-1 is transmitted on theboundary after CMD #4 (a delay of only 1 packet length). In addition,the controller sampling clocks can be different for each rank tocompensate for the skew. As shown, the rank-0 sampling clock and therank-1 sampling clock may have different frequencies and/or be out ofphase.

6 Clocking and Datapath 6.1 Overview

This section provides a brief discussion on the clocking and datapathtopology used in the ARAM interface architecture. FIG. 18 is aconceptual diagram illustrating an overview of the topologies utilized.Note that it is a Flexphase-based system where the CPHY PLL generates a1 GHz clock from a low speed reference. This clock is used by the Rx,Tx, serializing and deserializing circuitry within the CPHY, and isforwarded to the ARAM PHY for use there as a sampling and transmit clockas well as a source for the framing clock.

6.2 ARAM Device Clocking

The CK link signal forwarded from the controller PHY supplies theclocking waveform for the ARAM device. The signal comes indifferentially, using VLS NG levels. It is amplified to CMOS levels andthen distributed across the interface. A divided-down version of thisclock supplies the framing information for the serialization anddeserialization function, while another version of the divided clock issent to the Link Layer as its clock source. FIG. 19 illustrates aconceptual diagram of the internal clocking of an ARAM device, accordingto an embodiment. Note that the CK and CA input delay paths need to bematched as indicated on the diagram. This can allow the CPHY tocommunicate with all devices on its CA link without having to make aper-device CA-vs-CK phase adjustment, and is important for optimizinglatency and efficiency of the command link.

FIG. 20 shows a conceptual diagram of the clocking internal to the ARAMCPHY, according to an embodiment. As indicated previously, the CPHY PLLtakes a reference clock as an input (100/250 MHz) and multiplies it tocreate its 1 GHz output multi-phase clocks. A version of these clocks,through a phase mixer, is forwarded to the channel and eventually theARAM device. The multi-phase clocks also serve as the input to the perDQ phase mixers which will generate the per DQ high speed clocks, usedto clock the Rx/Tx/Serializer/Deserializer circuitry. Calibration atstartup ensures that the phase placement of these clocks allow forsufficient and robust timing margins on the ARAM interface channel. ADLL serves to help synchronize the clock boundary crossing between theCPHY clock domain and the AMC clock domain (represented by the PCLKsignal in the diagram). By retiming the data across the domains usingthe DLL and its output, latencies can be kept consistent and minimized.The PLL also needs to support a low-speed mode (e.g. REFCLK/2 mode) toenable communication with ARAM devices before the ARAM interface channelis fully calibrated at startup.

6.4 Calibration

The steps needed to calibrate the clocks within the ARAM interfacearchitecture are discussed in the Initialization and Calibrationsection.

6.5 Starting and Stopping CK

When enabling/disabling an ARAM PHY through the EN signal orstarting/stopping the forwarded CK link, it is important to take carenot to create any glitches on the link which can propagate through theattached ARAM devices.

When starting or stopping the forwarded CK/CKB signals, the CPHY mustassure that no runt pulses propagate down the channel. Any such pulsescan corrupt the divider state within the ARAM PHY and could possiblycause a undefined clock to enter the logic blocks of the device. Inorder to meet this requirement, CPHY uses a synchronous clock-gatingcircuit to stop and start forwarded clock CK/CKB signals. Because ENsignal assertion/deassertion is asynchronous and may happen when theforwarded clock is running (in multi-rank case for example), the ARAMPHY must guarantee that the glitches or runt pulses happen at its clockdivider outputs. When the EN signal is deasserted to put an ARAM devicein a low-power state, the ARAM PHY in the ARAM device sets its clockdivider outputs to a known static state without glitch and then powersdown its bias circuits including the bias circuit for the forwardedclock amplifiers. When EN signal is asserted, the bias circuits areenabled, the ARAM PHY will then wait for a CA preamble to restart itsclock dividers.

7 Register Operations

The ARAM device registers are accessible in the INIT and ACTIVE modes.They can be accessed at BOOTMODE rates at any point, but can also beaccessed at the full rate after the CA and DQ links are calibrated.

Writing to the registers can be executed entirely through the CA commandpacket, utilizing the WRT_REG command along with its payload of registeraddress and contents.

Reading from the registers requires use of the CA link as well as the DQlink. The architecture has some flexibility as to which DQs will deliverthe register contents, and this can be determined during implementationor can be made configurable.

8 Power-On, Initialization and Calibration 8.1 Overview

Before a powered down ARAM PHY can be used in normal operating mode, itmay need to be powered-on and calibrated. The power-on and initialcalibration sequence of an ARAM PHY may occur as follows:

1. Power-On Reset/INIT state

2. Device ID and Dynamic Width Assignment

3. VREF and Output Impedance Calibration

4. CLK DCC Calibration

5. CA Phase Calibration

6. DQ READ Phase Calibration and Leveling

7. DQ WRITE Phase Calibration and Leveling

8.2 Power-On Reset/INIT mode:

As the supply regulators are being turned-on and the supply voltages areramping, EN can be held low, while CK and the CA inputs must be held ata valid VLS-NG level. The DQ drivers power up in high impedance.

The power-on block within the interface will generate a self-timed resetpulse as the interface supply (VDD) finishes ramping. It is assumed thatonce the supplies are turned on, their final output voltages will bereached within tINIT0<a threshold amount of time. This reset pulse canplace the ARAM PHY and Link Layer in their standard power-on resetcondition which will be referred to as the INIT mode.

By asserting the EN signal after reset is complete, the PHY can beginoperation at BOOTMODE datarates/frequencies.

Within the INIT state, all of the calibration commands will be enabled,allowing for any or all of the ARAM PHY calibration steps to beinitiated. Other than Power-on reset, the INIT MODE command can take thedevice from normal operation into INIT mode, while the command EXIT INITreturns the device to standard operation. Other implementation optionsinclude using Register Writes to transition the device in and out ofINIT mode

8.3 Device ID and Dynamic Width Assignment

After power-on reset has completed, in order to avoid the possibility ofdriver conflict, the controller can configure each of the ranksindividually. Until all devices in all ranks have been configured, onlyone EN signal can be asserted at any one time.

When its EN signal is asserted, an ARAM PHY coming out of power-on resetis able to receive the CLK signal and decode commands from its CA inputsat BOOTMODE datarates. By loading a known pattern into internalregisters and then reading it back out onto the devices, the system'sconnectivity can be determined, and each ARAM device can be assigned aDevice ID. In addition, any dynamic width settings can also be assignedto each individual device at this time.

8.4 VREF and Output Impedance Calibration

After the assignment of DeviceIDs and Dynamic width settings, it ispossible to initiate calibration of the reference voltage VREF and theoutput impedance on each device.

By sending a VREF_CAL command to a specific ARAM device, the controllercan initiate VREF calibration on that device. Upon receiving anddecoding this command, the ARAM Link Layer will enable the DQ receiversin the PHY and inhibit decoding of CA commands while it executes thisVREF calibration command. The controller PHY will then set its VEXTvoltage to its nominal (normal operation) level, followed by initiatingits VREF calibration procedure. After the predetermined number of cycleshave passed, the controller will stop sending its calibration procedureand the ARAM device's Link Layer will re-initiate its decoding of CAcommands.

By sending a UP_IMP_CAL command to an ARAM device, the controller caninitiate a pull-up impedance calibration command. The controller needsto set its DQ termination at the target impedance—e.g. 40 ohms—toground. Upon receiving and decoding this command, the ARAM Link Layerwill enable the DQ transmitters and have them transmit a DC (static)logical “1”. By comparing the resulting voltage at the ARAM DQ pins toVEXT/2 the ARAM can adjust the pull-up impedance setting until it isproperly calibrated to the desired value.

Similarly, by sending a DN_IMP_CAL command to an ARAM device, thecontroller can initiate a pull-down impedance calibration command. Thecontroller needs to set its DQ termination at the target impedance toVEXT. Upon receiving and decoding this command, the ARAM Link Layer willenable the DQ transmitters and have them transmit a DC (static) logical“0”. Comparing the resultant voltage at its DQ pins versus VEXT/2, theARAM can adjust the pull-down impedance setting until it is properlycalibrated to the desired target value.

8.5 Clock DCC Calibration

Calibrating out the duty cycle error in the clock distribution isinitiated similarly to other calibration procedures, however it is a bitmore involved since it requires a change in the frequency of operation(so that the duty cycle error with an “at-speed” clock can becorrected).

The procedure begins with the controller sending a CK_DCC command. TheARAM Link Layer, upon receiving and decoding this command, configuresthe ARAM PHY to measure the duty cycle of the internal clock waveform(and eventually send the measurement information to the Link Layer).After the PHY has been configured correctly, the controller will,without glitches, stop its clock (previously running at boot-mode rate)and restart it at the normal operation rate (e.g. 1 GHz). After aprescribed number of cycles (amounting to the time needed to do the DCCcalibration) the controller will again glitchlessly stop its clock andrestart it at the boot-mode frequency.

8.6 CA Phase Calibration

Now that the clock duty cycle has been calibrated, it is possible tobegin phase calibration for the CAs and DQs. Note that CA calibration isintended to compensate for system-level skew between CAs and Clock (theinternal delays associated with CK and CAs are assumed to be matched(i.e. CK and CAs are path matched internal to the ARAM device)).

CA calibration is initiated by the controller sending the CA_TCALcommand to the ARAM device. In response, the Link Layer will inhibit thedecoding of the packets on the CA link for a prescribed amount of time,and configure itself to compare the command payload to a knowncalibration pattern (previously stored in a register). In addition, theARAM will be configured to send the result of the comparison back to thecontroller (via a non-CA channel, e.g. DQ0).

Once the ARAM is configured, the controller will again stop the bootmoderate CK and then restart CK at the nominal frequency. At this point, astandard FlexPhase based calibration can be executed, thereby optimizingtiming for the CA link. After a prescribed amount of clock cycles, thephase calibration is assumed to have concluded and the ARAM willconfigure itself back to decoding CA packets.

8.7 DQ READ—Phase Calibration and Leveling

Discovery, address assignment, and dynamic width configuration have allbeen accomplished with static write DQ data. At this point, read DQleveling can be performed. This may be accomplished with burst readoperations of PRBS sequences, allowing the controller to determine theoptimal per-device FlexPhase timings.

8.8 DQ Write—Phase Calibration and Leveling

Once robust DQ read timings have been established, the per-rank timingcalibration of CA timing relative to CK can be performed. This can befacilitated by commands that request the ARAM device to return on the DQbus data that was received in the payload of the CA packet.

Finally, once optimal CA and read DQ timings have been established,write DQ timing calibrations can be performed. The readback of theresults of comparisons between write DQ data and expected data derivedfrom either command payloads or internal PRBS logic enablesdetermination of optimal controller per-rank, per-lane FlexPhasesettings.

After the initial configuration and calibration has been completed,periodic calibrations may be performed to track changes in operatingconditions as necessary.

8.9 ACK Phase Calibration

Each ACK link may require calibration to operate at its maximum rate. Asindicated in the section discussing the ACK protocol, there are skewsand delays that need to be accounted for to maximize receive timing forthe ACK link. Similar to the other links, these are compensated for byadjusting the FlexPhase settings controlling the receive clock phase atthe controller PHY.

The ACK link calibration can occur alongside the DQ calibration duringinitial calibration. By having all devices of a given rank transmittinga known pattern on its ACK link (at their assigned time slots), thecontroller will be able to discern a data eye, based on which thecontroller can adjust its ACK sampling clock edges to maximize timingmargin. This step may need to be repeated for each rank (or can be doneconcurrently).

8.10 Additional Information 8.10.1 Device ID and Dynamic WidthAssignment

A discussion of possible implementations of device identification anddynamic width assignment follows below. While complexities of solutionsvary, what follows is believed to be a good compromise between systemflexibility and straightforward design.

8.10.1.1 System topology, Device ID, and Dynamic Width Configuration

In many applications which utilize ARAM, the controller will not haveaccess to an additional non-volatile memory resource that describes therank and dynamic width connectivity of the attached ARAM devices. Anexample sequence of ARAM read and write operations and ARAM registerresources that can be used for discovery is described in Section8.10.1.2.

It is assumed that there is no persistent or non-volatile storage withinthe ARAM device that holds the device address mask field used to qualifyCA packets. An example sequence of write operations and ARAM registerresources that accomplish device address mask assignment are describedin Section 8.10.1.3.

Finally, once connectivity of devices within a rank has been discoveredand device address masks have been assigned, the required Dynamic WidthDQ datapath de-multiplexing and multiplexing between the ARAM PHY andLink layers must be configured. An example sequence of operations thataccomplishes this is described in Section 8.10.1.7.

8.10.1.2 System Topology

Without giving up the capacity scalability benefits of dynamic width,substantial reductions in the total number of operations and theconditional complexity that the controller must execute can be achievedby restricting the total number of allowed configurations. Reductions inthe total amount of ARAM link layer datapath multiplexing will alsoresult from restricting the possible configurations. For this reason, itis proposed that only the three connectivities on the DQ link shown inTable 6 be allowed:

TABLE 6 Reduced Set of Allowed Connectivities Controller 1 ARAM/rank 2ARAMs/rank 4 ARAMs/rank DQ[3] ARAM0 DQ[3] ARAM1 DQ[1] ARAM3 DQ[0] DQ[2]ARAM0 DQ[2] ARAM1 DQ[0] ARAM2 DQ[0] DQ[1] ARAM0 DQ[1] ARAM0 DQ[1] ARAM1DQ[0] DQ[0] ARAM0 DQ[0] ARAM0 DQ[0] ARAM0 DQ[0]

TABLE 7 Read Responses for Reduced Set of Allowed Connectivities 1 ARAM2 ARAMs 4 ARAMs DQ = 4′b0001 DQ = 4′b0101 DQ = 4′b1111

8.10.1.3 Device Address Assignment

In order to enable individual devices to respond to the broadcast CApackets, each ARAM device needs to be assigned a unique address maskfield. This is accomplished by establishing unique correlations betweeninformation that is broadcast to all devices within CA payloads andinformation that is received by individual devices on their DQ pins.

It is assumed that all devices will respond to the command sent in a CApacket with CADDR[7:0] 8′h00, which provides a way to communicate withuninitialized and unconfigured devices. CADDR[7:0] also serves as thebroadcast address.

It is proposed that the CA packet address field CADDR[7:0] be logicallysplit in to two subfields, CARANK and CADEVICE, whereCADDR[7:0]={CARANK[3:0], CADEVICE[3:0]}. Splitting the CADDR this waywill support configurations of one to four ranks, with up to fourdevices per rank. Commands targeted for individual ranks, sets of ranks,and individual devices or sets of devices within ranks are addressedthrough one-hot encodings.

Within each ARAM device there is an 8-bit ADDRMASK register. CA packetsare accepted by a device when (|(CADDR[7:4] & ADDRMASK[7:4]) &|(CADDR[3:0] & ADDRMASK[3:0]))|(CADDR==8′h00) is true.

In addition to the ADDRMASK register, device address assignment can beaccomplished by populating a temporary results register TEMP with theresults of a logical operation performed on CMD payload DQ data, andthen copying the contents of TEMP to ADDRMASK. Specific details of howto assign device addresses for the three recommended (1 device, 2device, and 4 device per rank) connectivity configurations follows.

8.10.1.4 Single x4 Device/Rank Address Assignment

Assigning an address to single device entirely populating a rank isstraightforward. The controller will first broadcast a commandWRITE_CP_TEMP that instructs the ARAM to write the desired RANK andDEVICE mask bits from the CMD Payload to TEMP. The controller will thenissue a WRITE_TEMP_ADDRMASK command, which results in the ARAM devicecopying the contents of the TEMP register to the ADDRMASK register,committing the address assignment.

8.10.1.5 Dual x2 Dynamic Width Device/Rank Address Assignment

First, the TEMP register is initialized in each device by broadcasting aWRITE_CP_TEMP command with the CMD Payload==8′b{RANK[3:0], 4′b0000},where RANK denotes the desired RANK. This is followed by a broadcastWRITE_CP_AND_DQ_TEMPL command, where the static value 4′b1001 is drivenon the DQ bus, and the 4 least significant bits of the CMD Payload (CP)4′b0011. As a result of this operation the ARAM can setTEMP[3:0]=CP[3:0] & DQ[3:0]. The ARAM device connected to controllerDQ[3:2] device will have TEMP[3:0]=(4′b0011 & 4′bxx10), while the deviceconnected to controller DQ[1:0] will have TEMP[3:0]=(4′b0011 & 4′b0001).Address assignment is then completed with the WRITE_TEMP_ADDRMASKcommand.

8.10.1.6 Quad x1 Dynamic Width Device/Rank Address Assignment

In this configuration, all ARAM devices will observe the DQ linkidentically. In order to assign unique device addresses, the contents ofeach devices TEMP register will need to be assembled one bit at a time.The bit-by-bit construction can be accomplished by specifying in the CMDPayload which bit of the TEMP register is to be set if there is a matchbetween the broadcast CA pattern and the observed DQ. Execution of theWRITE_CP_AND_DQ_TEMP_n command will result in the followingTEMP[n]=|(CP[3:0] & DQ[3:0]) where both n and CP[3:0] are broadcast inthe CA command payload. The complete sequence of broadcast (CADDR=8′h00)commands, command payload fields, and necessary to assign addresseswithin the enabled rank to four devices follows:

WRITE_CP_TEMP CP==8′b{RANK[3:0], 4′b0000}

WRITE_CP_AND_DQ_TEMP_n CP {3′b000, 4′b0001}, DQ==4′b0001(TEMP[0]=|(4′b0001 & DQ[3:0])

WRITE_CP_AND_DQ_TEMP_n CP {3′b001, 4′b0001}, DQ==4′b0010(TEMP[1]=|(4′b0001 & DQ[3:0])

WRITE_CP_AND_DQ_TEMP_n CP {3′b010, 4′b0001}, DQ==4′b0100(TEMP[2]=|(4′b0001 & DQ[3:0])

WRITE_CP_AND_DQ_TEMP_n CP {3′b011, 4′b0001}, DQ==4′b1000(TEMP[3]=|(4′b0001 & DQ[3:0])

WRITE_TEMP_ADDRMASK

It is assumed that power-on detection will initialize the ARAM devicesinto full native width mode (x4 mode for x4 devices, x8 for x8 devices).Once connectivity discovery and address assignment has been completed,each ARAM device can be configured to the appropriate DQ datapath muxwidth with a WRITE_CP_DQWIDTH command, where DQWIDTH=CP[3:0].

If the ARAMs include a full crossbar that allows complete flexibility ofdatapath routing between the PHY DQ and the link layer that need to beconfigured, this can be accomplished with a sequence of commands which,in their payload field, specify the required PHY DQ to link DQ mapping.

It is assumed that although an ARAM has been configured to a reduced DQwidth, the granularity of accesses will remain unchanged. That is, whenDQWIDTH=4′b0010 (x2 mode), each byte from a device will be serializedacross 4 CK cycles.

Although there are a few possible implementations, the following isdesigned to cancel out any AC related variation and hence produce anaccurate effective VREF value. After the controller sets VEXT to thedesired value, it begins to transmit a half-rate clock pattern to theARAM DQs. Using a low-pass filter, the ARAM must extract the common modeof the incoming signals and utilize that as the desired VREF value. TheARAM must compare this desired value to the internal VREF that it isusing, and the Link Layer must adjust the setting on the internal VREFgenerator to move its voltage closer to the desired voltage. This stepwould iterate until the internal voltage has been adjusted to the samevalue as the desired target.

The power supplies ramp up in this sequence: VDDP, VDD, VDDIO, whereVDDP (if necessary) is a boosted voltage power supply for ARAM core highpower or high voltage resources, VDD is the power supply for the ARAMPHY and Link layer CMOS logic, and VDDIO is the supply used for theVLS-NG drivers and receivers.

The detection of the ramp of power (VDD and VDDIO, in particular) mustinitialize enough state within the ARAM PHY and Link layers to enter thePHY Powerdown state, and respond to subsequent CA packets once VEXT hasstabilized and EN is enabled. At a minimum, this would entail makingsure that at least one pair (rank and device) of CA packet address maskbits are enabled if the ARAM device does not implement detection ofCADDR=8′h00 as an unconditional match. 8.10.1.7 Dynamic WidthConfiguration 8.10.1.8 VREF and Impedance Calibration

9 Low-Speed Mode

Low speed mode can be utilized whenever the ARAM interface channel hasnot been calibrated for high speed operation, or at any time is in needof recalibration. With care, it can also be utilized even if the ARAMinterface is calibrated, if low speed operation is desired.

Note that prior to the completion of initialization, CA write leveling,and DQ calibration, the controller will launch CA data relative to oneedge of CK while the ARAM will sample CA with the opposite edge of CK,nominally providing maximum setup and hold margin for commands. Alsonote that initialization can be accomplished with static values beingexchanged on the DQ bus, as described in Section 8.

To maintain framing synchronization between the controller and ARAM,it's probably best to de-assert and re-assert EN immediately afterchanging the frequency of CK, as the first CA packet received after theassertion of EN will be used to re-establish ARAM framing.

10 ARAM PHY and Interface Operating States

The ARAM interface architecture supports a means to powerdown the ARAMPHY for flexible power management. The operating states and the activityof the associated PHY power domains are described in Table 8.

TABLE 8 PHY and Interface Operating States Functional State EN OperatingState Poweroff N/A No supply Cal_Init 0 Powerup-reset enters this state.PHY is disabled by EN. Enabling the PHY places interface in the Cal_initCal_Init 1 Calibration and Initialization state All calibration andinitialization commands are accessible in this state Ready 0 PHY isdisabled by EN. Enabling the PHY places interface in the Ready state(assumes calibration is valid). Ready 1 Standard, calibrated operatingmode.

10.1 ARAM PHY Operating State Transitions

The transitions between the different operating states are shown in FIG.21. During when the ARAM device is powered on, a power-on reset circuitwill reset the state of the ARAM PHY and any additional circuitrynecessary to enter the Cal_Init state. When the EN signal is asserted,the PHY will be able to receive and execute the necessary initializationand calibration commands from the CPHY.

The READY_ST command will place the PHY into the READY state, able todecode the standard CA commands. Similarly, the CAL_INIT_ST command willplace the PHY into the Cal_Init state, able to accept any initializationand calibration commands. The controller needs to keep track of whethercalibration is valid in either state and whether high data rateoperation is possible or not.

The EN signal enables and disables the ARAM PHY, and hence needs to beasserted any time clocks are required within the ARAM device (includingthe Link Layer).

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general-purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct a more specializedapparatus to perform the operations. The required structure for avariety of these systems will appear from the description below. Inaddition, the present embodiments are not described with reference toany particular programming language. It will be appreciated that avariety of programming languages may be used to implement the teachingsof the embodiments as described herein.

The above description sets forth numerous specific details such asexamples of specific systems, components, methods and so forth, in orderto provide a good understanding of several embodiments. It will beapparent to one skilled in the art, however, that at least someembodiments may be practiced without these specific details. In otherinstances, well-known components or methods are not described in detailor are presented in simple block diagram format in order to avoidunnecessarily obscuring the present embodiments. Thus, the specificdetails set forth above are merely exemplary. Particular implementationsmay vary from these exemplary details and still be contemplated to bewithin the scope of the present embodiments.

It is to be understood that the above description is intended to beillustrative and not restrictive. Many other embodiments will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the present embodiments should,therefore, be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

1. (canceled)
 2. A memory controller comprising: a command interface totransmit a memory command to a plurality of memory devices associatedwith the memory controller; an acknowledgement interface to receive acombined acknowledgment status packet from the plurality of memorydevices over a shared acknowledgement link coupled between the memorycontroller and the plurality of memory devices, wherein the combinedacknowledgment status is formed from a logical combination of individualacknowledgment status packets corresponding to each of the plurality ofmemory devices, wherein each of the individual acknowledgment statuspackets comprises a plurality of fields, and wherein each of theindividual acknowledgement status packets comprises acknowledgmentinformation associated with a corresponding memory device of theplurality of memory devices and stored in a different field of theplurality of fields, the acknowledgment information to indicate whetherthe command was successfully received by the corresponding memorydevice; and a memory controller core to decode the combinedacknowledgment status packet to identify the acknowledgement informationcorresponding to each of the plurality of memory devices.
 3. The memorycontroller of claim 2, wherein each of the individual acknowledgmentstatus packets comprises one field comprising acknowledgment informationassociated with the corresponding memory device and a plurality offields comprising default values.
 4. The memory controller of claim 3,wherein the one field comprises at least one of a first value indicatingthat the command was successfully received by the corresponding memorydevice or a second value indicating that the command was notsuccessfully received by the corresponding memory device.
 5. The memorycontroller of claim 4, wherein the combined acknowledgment status packetcomprises a plurality of fields each comprising a logical combination ofone field comprising acknowledgment information from one of theindividual acknowledgement status packets and one or more fieldscomprising default values from one or more other individualacknowledgment status packets.
 6. The memory controller of claim 5,wherein to decode the combined acknowledgment status packet, the memorycontroller core is to determine which field of the plurality of fieldsin the combined acknowledgment status packet is associated with each ofthe plurality of memory devices.
 7. The memory controller of claim 2,wherein the logical combination is performed by a wired-OR connection.8. The memory controller of claim 2, wherein the plurality of memorydevices comprises advanced random access memory (ARAM) devices.
 9. Amethod comprising: transmitting, by a memory controller, a memorycommand to a plurality of memory devices; receiving, by a memorycontroller, a combined acknowledgment status packet from the pluralityof memory devices over a shared acknowledgement link coupled between thememory controller and the plurality of memory devices, wherein thecombined acknowledgment status is formed from a logical combination ofindividual acknowledgment status packets corresponding to each of theplurality of memory devices, wherein each of the individualacknowledgment status packets comprises a plurality of fields, andwherein each of the individual acknowledgement status packets comprisesacknowledgment information associated with a corresponding memory deviceof the plurality of memory devices and stored in a different field ofthe plurality of fields, the acknowledgment information to indicatewhether the command was successfully received by the correspondingmemory device; and decoding, by the memory controller, the combinedacknowledgment status packet to identify the acknowledgement informationcorresponding to each of the plurality of memory devices.
 10. The methodof claim 9, wherein each of the individual acknowledgment status packetscomprises one field comprising acknowledgment information associatedwith the corresponding memory device and a plurality of fieldscomprising default values.
 11. The method of claim 10, wherein the onefield comprises at least one of a first value indicating that thecommand was successfully received by the corresponding memory device ora second value indicating that the command was not successfully receivedby the corresponding memory device.
 12. The method of claim 11, whereinthe combined acknowledgment status packet comprises a plurality offields each comprising a logical combination of one field comprisingacknowledgment information from one of the individual acknowledgementstatus packets and one or more fields comprising default values from oneor more other individual acknowledgment status packets.
 13. The methodof claim 12, wherein to decode the combined acknowledgment statuspacket, the memory controller core is to determine which field of theplurality of fields in the combined acknowledgment status packet isassociated with each of the plurality of memory devices.
 14. The methodof claim 9, wherein the logical combination is performed by a wired-ORconnection.
 15. The method of claim 9, wherein the plurality of memorydevices comprises advanced random access memory (ARAM) devices.
 16. Amemory device comprising: a command interface to receive a memorycommand from a memory controller associated with the memory device; andan acknowledgment interface to generate and transmit a combinedacknowledgment status packet to the memory controller over a sharedacknowledgment link coupled between the memory controller and the memorydevice, wherein the acknowledgment link is shareable by a plurality ofmemory devices associated with the memory controller, wherein thecombined acknowledgment status packet is formed from a logicalcombination of individual acknowledgment status packets corresponding toeach of the plurality of memory devices, wherein each of the individualacknowledgment status packets comprises a plurality of fields, andwherein each of the individual acknowledgement status packets comprisesacknowledgment information associated with a corresponding memory deviceof the plurality of memory devices and stored in a different field ofthe plurality of fields, the acknowledgment information to indicatewhether the command was successfully received by the correspondingmemory device.
 17. The memory device of claim 16, wherein each of theindividual acknowledgment status packets comprises one field comprisingacknowledgment information associated with the corresponding memorydevice and a plurality of fields comprising default values.
 18. Thememory device of claim 17, wherein the one field comprises at least oneof a first value indicating that the command was successfully receivedby the corresponding memory device or a second value indicating that thecommand was not successfully received by the corresponding memorydevice.
 19. The memory device of claim 18, wherein the combinedacknowledgment status packet comprises a plurality of fields eachcomprising a logical combination of one field comprising acknowledgmentinformation from one of the individual acknowledgement status packetsand one or more fields comprising default values from one or more otherindividual acknowledgment status packets.
 20. The memory device of claim16, wherein the logical combination is performed by a wired-ORconnection.
 21. The memory device of claim 16, wherein the plurality ofmemory devices comprises advanced random access memory (ARAM) devices.